Method of removing a blocking phenomenon in a first block using properties of second and third blocks adjacent the first block

ABSTRACT

In one embodiment, the method includes obtaining a block type of the second block and the third block. The second block is adjacent to the first block horizontally, and the third block is adjacent to the first block vertically. The method further includes determining whether the second and third blocks have non-zero transform coefficient information if the block type of the second and third block, respectively, are not an intra-coded blocks. First and second motion vector differences between a motion vector for the first block and motion vectors for the second and third blocks, respectively, are obtained. A horizontal filter strength value and a vertical filter strength value are determined based on the above steps, and the blocking phenomenon in the first block is removed according to the horizontal filter strength value and the vertical filter strength value.

DOMESTIC PRIORITY INFORMATION

This is a divisional of U.S. application Ser. No. 11/097,394 filed Apr.5, 2005, which is a divisional of U.S. application Ser. No. 10/170,999filed Jun. 14, 2002, now issued as U.S. Pat. No. 7,272,186; the contentsof all of which are hereby incorporated by reference in their entirety.

FOREIGN PRIORITY INFORMATION

The present invention claims priority under 35 U.S.C. 119 on KoreanApplication No. 2001-33953 filed Jun. 15, 2001; the contents of whichare hereby incorporated by reference in their entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a moving picture compression technique,and more particularly, to a loop filtering method in a video coder.

2. Description of the Background Art

Generally, it has been proved that the “H.26L” system developed as anext generation moving picture compression technique is superior to the“H.263” and “MPEG4” systems in performance.

The “H.26L” system as the next generation moving picture compressionsystem differs from the conventional H.263 and MPEG4 in using 4×4 blockbased transformation & encoding, carrying out motion estimation &compensation of a transformation block size, and using a single variablelength coder (VLC).

Superior to the conventional motion picture standards in aspect ofperformance, the H.26L system has excessive calculation of encoder aswell as uses 4×4 block based transformation so that a blocking dominatesthan the conventional standards.

In order to remove the blocking, filters include a loop filtering systeminside the encoder and decoder, a post filter processed in a next stageof the decoder, and a system adding the loop filtering system to thepost filter.

When the blocking is removed by the loop filtering system in general, adifference between an input video and a motion video is affected so thatan encoding rate increases in a specific case. Yet, as the loopfiltering system is installed inside a video coder, complexity of theencoder increases.

As mentioned in the above explanation, the H.26L system according to therelated art requires excessive quantity of calculation despite thesuperiority in performance to the moving picture standards as well ashaving the blocking more dominant than that of the moving picturestandards due to the 4×4 block based transformation system.

Moreover, if the blocking is removed by the loop filtering systemaccording to the related art, the difference between the input video andthe motion video is affected so as to increase the encoding rate in aspecific case. Yet, the loop filtering system built inside the videocoder increases the complexity of the encoder.

SUMMARY OF THE INVENTION

The present invention relates to a method of removing a blockingphenomenon in a first block using properties of a second block and athird block, where the second and third blocks are adjacent the firstblock.

In one embodiment, the method includes obtaining a block type of thesecond block and the third block. The second block is adjacent to thefirst block horizontally, and the third block is adjacent to the firstblock vertically. The method further includes determining whether thesecond block has non-zero transform coefficient information if the blocktype of the second block is not an intra-coded block, and determiningwhether the third block has non-zero transform coefficient informationif the block type of the third block is not an intra-coded block. Themethod still further includes obtaining a first motion vector differencebetween a motion vector for the first block and a motion vector for thesecond block, and obtaining a second motion vector difference between amotion vector for the first block and a motion vector for the thirdblock. The method also includes determining a horizontal filter strengthvalue based on the block type of the second block, the determination ofwhether the second block has non-zero transform coefficient information,and the first motion vector difference. A vertical filter strength valueis determined based on the block type of the third block, determinationof whether the third block has non-zero transform coefficientinformation, and the second motion vector difference. The blockingphenomenon in the first block is removed according to the horizontalfilter strength value and the vertical filter strength value.

It is to be understood that both the foregoing general description andthe following detailed description of the present invention areexemplary and explanatory and are intended to provide furtherexplanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIG. 1 illustrates a flowchart of a loop filtering process according toan embodiment of the present invention;

FIGS. 2A and 2B illustrate block diagrams indicating block locations inhorizontal and vertical directions according to an embodiment of thepresent invention, respectively;

FIG. 3 illustrates a diagram of pixel locations of adjacent blocksaccording to an embodiment of the present invention; and

FIG. 4 illustrates a table of LQP values for determining DQP(QP) valuesaccording to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE EXAMPLE EMBODIMENTS

Reference will now be made in detail to the example embodiments of thepresent invention, which are illustrated in the accompanying drawings.

Hereinafter, a loop filtering method in a video coder according to anexample embodiment of the present invention carries out a real-timeprocessing of removing a blocking and a ringing phenomenon in accordancewith a pixel location, a coded block pattern, a quantization step size,and a motion vector size in a digital video appliance using the H.26Lmoving picture compression technique, thereby enabling to improve avideo quality of a compression image demanding a low bit rate orhigh-speed processing.

First, explained in detail is a real-time processing technique ofremoving “blocking” and “ringing” by predicting an original video forcorrelation between pixels in accordance with a quantization variableQP, a coded block pattern, and a motion vector size available for adecoder (not shown in the drawing).

When an original video f is transmitted as a compressed form, a videorestructured in a decoder can be expressed by the following Formula 1-1.

[Formula 1-1]

g=f+n, where g, f, and n are a compression video rearranged in ascanning order, an original video, and a column vector of quantizationerror, respectively.

A relation between respective pixels can be expressed by the followingFormula 1-2 in order to process the Formula 1-1 by pixel unit.

[Formula 1-2]

g(i,j)=f(i,j)+n(i,j), where i and j indicate a location of a pixel invertical and horizontal directions.

An additional function can be defined as the following Formula 1-3 whenreliability of original pixels and 1-dimensional smoothing function foreach pixel are used together with a regularization. Firstly, ahorizontal direction is considered.

[Formula 1-3]

M{f(i,j)}=M_(p){f(i,j)}+M_(N){f(i,j)}, where M_(P) and M_(N) arefunctions representing reliability smoothing degrees of left and rightdirections for a pixel f(i,j), respectively.

And, such functions are defined as the following Formula 1-4, where theM_(P) and M_(N) are the functions considering f(i,j) & f(i,j−1) andf(i,j) & f(i,j+1), respectively.M _(P) {f(i,j)}=α_(P) {f(i,j)−f(i,j−1)}²+(1−α_(P)){g(i,j)−f(i,j)}²M _(N) {f(i,j)}=α_(N){f(i,j)−f(i,j−1)}²+(1−α_(N)){g(i,j)−f(i,j)}²  [Formula 1-4]

Each of the first terms of right sides in Formula 1-4 indicatesirregularity with an adjacent pixel, i.e. a difference from an adjacentpixel, for the pixel f(i,j), and each of the second terms indicates areliability for the original pixel f(i,j). Moreover, α_(P) and α_(N)mean regularization parameters representing ratios between irregular andreliability terms defined in the functions, respectively. And, α_(P) andα_(N) are parameters adjusting a relative coherence between the twoterms in Formula 1-4.

With the above manner, the additional function is defined for each pixelof the moving picture, and the function in a vertical direction isdefined by varying the parameter i instead of the parameter j in Formula1-4.

Thereafter, if a differential is applied to the additional functions inFormula 1-4 defined for each pixel of the moving picture in order tofind a restored video from which the blocking and ringing are removed,the following Formulas 1-5 and 1-6 are attained. $\begin{matrix}{\frac{{\partial M}\{ {f( {i,j} )} \}}{\partial{f( {i,j} )}} = {{\frac{{\partial M_{P}}\{ {f( {i,j} )} \}}{\partial{f( {i,j} )}} + \frac{{\partial M_{N}}\{ {f( {i,j} )} \}}{\partial{f( {i,j} )}}} = 0}} & \lbrack {{Formula}\quad 1\text{-}5} \rbrack\end{matrix}$

From Formula 1-5, the following Formula 1-6 is found. $\begin{matrix}{{\frac{{\partial M_{P}}\{ {f( {i,j} )} \}}{\partial{f( {i,j} )}} = {{2{\partial_{P}\{ {{f( {i,j} )} - {f( {i,{j - 1}} )}} \}}} - {2( {1 - \alpha_{P}} )\{ {{g( {i,j} )} - {f( {i,j} )}} \}}}}{\frac{{\partial M_{N}}\{ {f( {i,j} )} \}}{\partial{f( {i,j} )}} = {{2{\partial_{N}\{ {{f( {i,j} )} - {f( {i,{j - 1}} )}} \}}} - {2( {1 - \alpha_{N}} )\{ {{g( {i,j} )} - {f( {i,j} )}} \}}}}} & \lbrack {{Formula}\quad 1\text{-}6} \rbrack\end{matrix}$

From Formulas 1-5 and 1-6, the pixel to be restored in a horizontaldirection can be found by the following Formula 1-7. $\begin{matrix}{{f( {i,j} )} = \frac{\begin{matrix}{{( {2 - \alpha_{P} - \alpha_{N}} ){g( {i,j} )}} +} \\{{\alpha_{P}{f( {i,{j - 1}} )}} + {\alpha_{N}{f( {i,{j + 1}} )}}}\end{matrix}}{2}} & \lbrack {{Formula}\quad 1\text{-}7} \rbrack\end{matrix}$

As the regularization parameters in Formula 1-7 are between “0” and “1”,Formula 1-7 can be defined by the following Formula 1-8. $\begin{matrix}{{f( {i,j} )} = \frac{\begin{matrix}{{( {2 - \alpha_{P} - \alpha_{N}} ){g( {i,j} )}} +} \\{{\alpha_{P}{g( {i,{j - 1}} )}} + {\alpha_{N}{g( {i,{j + 1}} )}}}\end{matrix}}{2}} & \lbrack {{Formula}\quad 1\text{-}8} \rbrack\end{matrix}$

Looking into Formula 1-8, a blocking-removed video of the compressionvideo is determined by the two, left and right pixel values and theregularization parameters by taking the i_(th) and j_(th) videos asreferences.

As the two, left and right pixel values can be used for encoding anddecoding, two regularization parameter values are set up for attainingthe restored video. For this, “set theoretic” is applied to therestoration by pixel unit so as to represent regularization coefficientsby the following Formula 1-9, where “set theoretic” is a theory forcontrolling the parameters α_(P) and α_(N) in Formula 1-4.$\begin{matrix}{{\frac{\alpha_{P}}{1 - \alpha_{P}} = {\frac{\{ {{f( {i,j} )} - {g( {i,j} )}} \}^{2}}{\{ {{f( {i,j} )} - {f( {i,{j - 1}} )}} \}^{2}} \approx \frac{K_{P}{QP}^{2}}{\{ {{g( {i,j} )} - {g( {i,{j - 1}} )}} \}^{2}}}},{\frac{\alpha}{1 - \alpha_{N}} = {\frac{\{ {{f( {i,j} )} - {g( {i,j} )}} \}^{2}}{\{ {{f( {i,j} )} - {f( {i,{j + 1}} )}} \}^{2}} \approx \frac{K_{N}{QP}^{2}}{\{ {{g( {i,j} )} - {g( {i,{j + 1}} )}} \}^{2}}}}} & \lbrack {{Formula}\quad 1\text{-}9} \rbrack\end{matrix}$

K_(P) and K_(N) in Formula 1-9 are determined statistically as adaptiveparameters depending on a location of a pixel since the irregularitiesbetween pixels at a block boundary and an inner block side are differentfrom each other. And, “QP” is a quantization parameter of a macro blockto which the pixel f(i,j) belongs.

Even though the restored video from which blocking is removed usingFormulas 1-8 and 1-9, the value of the regularization parameter requiresa floating-point operation so as to become a problem in aspect ofcalculation quantity of the filter to remove the “blocking”.

Therefore, Formulas 1-8 and 1-9 are transformed into the followingFormulas 1-10 and 1-11 for integer operation. $\begin{matrix}{{f( {i,j} )} = \underset{\_}{\begin{matrix}{{( {2^{9} - {\alpha_{P} \times 2^{8}} - {\alpha_{N} \times 2^{8}}} ){g( {i,j} )}} +} \\{{\alpha_{P} \times 2^{8} \times {g( {i,{j - 1}} )}} + {\alpha_{N} \times 2^{8} \times {g( {i,{j + 1}} )}}}\end{matrix}}} & \lbrack {{Formula}\quad 1\text{-}10} \rbrack \\{{\beta_{P} = {{\alpha_{P} \times 2^{8}} = \frac{{2^{8} \times K_{P}}{QP}^{2}}{{K_{P}{QP}^{2}} + \{ {{g( {i,j} )} - {g( {i,{j - 1}} \}}^{2}} }}},{\beta_{N} = {{\alpha_{N} \times 2^{8}} = \frac{{2^{8} \times K_{N}}{QP}^{2}}{{K_{N}{QP}^{2}} + \{ {{g( {i,j} )} - {g( {i,{j - 1}} \}}^{2}} }}}} & \lbrack {{Formula}\quad 1\text{-}11} \rbrack\end{matrix}$

In this case, β_(P) and β_(N), as defined in Formula 1-11, are variablesattained by multiplying α_(P) and α_(N) by 2⁸, respectively. Namely,β_(P) and β_(N) are parameters for making α_(P) and α_(N) into integers.

In an algorithm according to the present invention, the regularizationparameters as integer forms using Formula 1-11 are generated, and thenstored as a look-up table form to be used.

Hence, Formula 1-10 for blocking filtering can be found by knowing thepixel f(i,j) and locations and quantization parameter values of twopixels adjacent to the pixel f(i,j).

Meanwhile, in the loop filtering process of the present invention, avideo of which blocking in a direction of a horizontal axis is removedby loop-filtering a block of a moving picture is attained, and then thevideo is loop-filtered in a direction of a vertical axis so as toprovide the video of which blocking in a direction of a vertical axis isremoved.

FIG. 1 illustrates a flowchart of a loop filtering process according toan embodiment of the present invention.

Referring to FIG. 1, a loop filtering method according to an embodimentof the present invention includes the steps of determining a strength ofa block to be processed in accordance with a coded pattern and aquantization step size of each 4×4 block when a block of a movingpicture is loop-filtered in directions of its horizontal and verticalaxes (S11), carrying out a loop filtering in accordance with thedetermined strength of the corresponding block (S12), and carrying outan additional loop filtering at a boundary region of a macro block sincea relatively big blocking exists at the boundary region of the macroblock (S13). This is explained in detail by referring to FIG. 2A, FIG.2B, and FIG. 3.

FIGS. 2A and 2B illustrate block diagrams indicating block locations inhorizontal and vertical directions according to an embodiment of thepresent invention, respectively, and FIG. 3 illustrates a diagram ofpixel locations of adjacent blocks according to an embodiment of thepresent invention, in which block locations in directions of horizontaland vertical axes are shown.

First of all, the step of determining the block strength St and afiltering state is explained as follows.

The step of determining the block strength St and a filtering state is aprocess for determining the H.26L system as standards, for which TML(test model long-term number 5) is in progress. In this case, the TMLcompresses a video by taking a 4×4 block as a reference.

The TML (test model long-term) moving picture compression system uses atechnique of removing spatial and temporal redundancy information froman encoder so as to transmit spatially & temporally compressedinformation and additional information required for decoding. Of course,in order to remove the redundancy information on a temporal domain, MPEG(moving picture expert group) and H.263 motion compensation systemaccording to the related art are used. The technique of removing theblocking using a basic mechanism of such a TML system is generallytreated by 4×4 block unit.

In this case, H.263 and H.26L are moving picture compression standardsproposed by ITU-T (international telecommunicationsunion-telecommunication).

The system for removing the blocking according to an embodiment of thepresent invention may follow the system of the related art. Yet, thefiltering state is determined by the following pre-treatment steps 1 to3 since the degree of the blocking differs in accordance with a codedblock pattern, a quantization step size, and a motion vector size.

[Step 1]

Strength St of each 4×4 block is initialized by ‘0’ (St=0).

[Step 2]

If each of the 4×4 blocks is an intra-coded block or has a non-zerotransform coefficient, St=max(St,2).

[Step 3]

If a motion vector difference between respective motion vectors for thetwo blocks block1 and block2 in FIG. 2(a) and FIG. 2(b) is equal to orgreater than ‘1’, St=Max(St, 1).

In this case, max(a,b) means a maximum value of ‘a’ and ‘b’. Namely,max(a,b) is a function for selecting a greater value from ‘a’ and ‘b’.

Moreover, after the strength St of each of the blocks has beendetermined, the loop filtering is carried out in accordance with thequantization size of each of the blocks.

For carrying out the loop filtering, the quantization sizes of the twoblocks block1 and block2 are as follows.

First case is that the quantization value of the block block1 is smallerthan ‘21’ and the strengths of the two blocks block1 and block2 are not‘0’. And, second case is that the quantization value of the block block1is greater than ‘20’ and the strength St of one of the two blocks block1and block2 is not ‘0’.

In this case, the filtering state is determined differently inaccordance with the quantization size because an adaptive treatment isperformed due to the dominating generation of blocking in case of agreat quantization size.

Therefore, when the conditions for the pre-treatment steps 1 to 3 aremet, the loop filtering process is carried out by the following manner.

First, pixel locations of two blocks block1 and block2 are shown in FIG.3. If the above-defined filtering state of each of the blocks issatisfied, values of pixels c, d, e, and f are found by the followingFormula 2-1 (for calculating filter coefficient values to be used inFormula 2-5).r1=reg{St ₁ ,QP,abs(b−c)},r2=reg{St ₁ ,QP,abs(c−d)},r3=reg{max(St ₁+1,St ₂+1),QP,abs(d−e)},r4=reg{St ₂ ,QP,abs(e−f)},r5=reg{St ₂ ,QP,abs(f−g)}  [Formula 2-1]

From Formula 2-1, reg(.) is defined by the following Formula 2-2.$\begin{matrix}{{{{reg}( {{St},{QP},{Diff}} )} = \frac{256 \times {M({St})} \times {{DQP}({QP})}}{{{Diff} \times {Diff}} + {{M({St})} \times {{DQP}({QP})}}}},} & \lbrack {{Formula}\quad 2\text{-}2} \rbrack\end{matrix}$where Diff is a random value and ‘abs’ indicates a function of absolutevalue.

And, M(St) is defined by the following Formula 2-3.M(St=0)=1/6,M(St=1)=4/16,M(St=2)=8/16,M(St=3)=1  {Formula 2-3}

And, DQP(QP) is explained by referring to FIG. 4 as follows.

FIG. 4 illustrates a table of LQP values for determining DQP(QP) valuesaccording to an embodiment of the present invention. Namely, DQP(QP) isdefined by the following Formula 2-4, and can be found by substitutingthe LQP values shown in the table of FIG. 4. LQP is a quantization tabledefined in H.26L, and DQP defines a quantization error range inaccordance with the quantization table defined in H.26L as defined byFormula 2-4. Namely, the LQP values are “620”, “553”, “429”, . . . ,“24”, and “22” if the quantization variables QP are “0”, “1”, “2”, . . ., “30”, and “31”, respectively. Namely, as the quantization variable (0,1, 2, . . . , 30, 31) increases sequentially, the LQP value decreaseswith a predetermined value (620, 553, 492, . . . , 24, and 22).$\begin{matrix}{{{DQP}({QP})} = \frac{676}{{LQP}({QP})}} & \lbrack {{Formula}\quad 2\text{-}4} \rbrack\end{matrix}$

Formula 1-11 is equivalent to Formula 2-2, and QP², K_(P) (or K_(N)),and {g(i,j)−g(i,j−1)} in Formula 1-11 are parameters equivalent toDQP(QP), M(St), and Diff in Formula 2-2, respectively.

Hence, the filtered values of the pixels c, d, e, and f are calculatedby the following Formula 2-5 using the defined parameters.c _(mod)={(512−r ₁ −r ₂)×c+r ₁ ×b+r ₂ ×d+256}>>9d _(mod)={(512−r ₂ −r ₃)×d+r ₂ ×c+r ₃ ×e+256}>>9e _(mod)={(512−r ₃ −r ₄)×c+r ₃ ×e+r ₄ ×f+256}>>9f _(mod)={(512−r ₄ −r ₅)×f+r ₄ ×e+r ₅ ×g+256}>>9,  [Formula 2-5]where “>>” means a bit shift, and C_(mod), d_(mod), e_(mod), and f_(mod)mean values of filtering c˜f having the blocking shown in FIG. 3.

Meanwhile, the result from the above-explained processing brings aboutan effect of removing the blocking and ringing with satisfaction. Yet,micro blocking still exists in the boundary region of the macro blockconstituted by 16×16 block unit. In order to settle the micro blocking,an additional filtering, as shown in Formula 2-6, is carried out on thepixel lying at the boundary region of the macro block.d _(mod)=(c _(mod)+13×d _(mod)+2×e _(mod)+8)>>4e _(mod)=(2×d _(mod)+13×e _(mod) +f _(mod)+8)>>4,if QP>16 and d and e are macro block boundary pixels.

Accordingly, the present invention carries out a real-time processing ofremoving a blocking and a ringing phenomenon in accordance with a pixellocation, a coded block pattern, a quantization step size, and a motionvector size in a digital video appliance using the H.26L moving picturecompression technique, thereby enabling to improve a video quality of acompression image demanding a low bit rate or high-speed processing.

The foregoing embodiments are merely exemplary and are not to beconstrued as limiting the present invention. The present teachings canbe readily applied to other types of apparatuses. The description of thepresent invention is intended to be illustrative, and not to limit thescope of the claims. Many alternatives, modifications, and variationswill be apparent to those skilled in the art.

1. A method of removing a blocking phenomenon in a first block usingproperties of a second block and a third block, the second and thirdblocks being adjacent the first block, comprising: obtaining a blocktype of the second block and the third block, the second block beingadjacent to the first block horizontally, the third block being adjacentto the first block vertically, determining whether the second block hasnon-zero transform coefficient information if the block type of thesecond block is not an intra-coded block, determining whether the thirdblock has non-zero transform coefficient information if the block typeof the third block is not an intra-coded block, obtaining a first motionvector difference between a motion vector for the first block and amotion vector for the second block, obtaining a second motion vectordifference between a motion vector for the first block and a motionvector for the third block, determining a horizontal filter strengthvalue based on the block type of the second block, the determination ofwhether the second block has non-zero transform coefficient information,and the first motion vector difference, determining a vertical filterstrength value based on the block type of the third block, thedetermination of whether the third block has non-zero transformcoefficient information, and the second motion vector difference, andremoving the blocking phenomenon in the first block according to thehorizontal filter strength value and the vertical filter strength value.